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i ABSTRACT , ^ 

C This Study investigates^the reliability and construct 

▼alidity of a group administered test of Piaget* sforial cperations 
stage. A related prctlei involving a learoing effect associated with 
Piaget* sWlinical lethods is also investigated. The Piagetian Icgical 
Operation Test (PLCT) » a g'roup-adiinistered instruient, lia's . 
deveIoped\nd field- tested. Eighty-foci students in grades 10-12 
participataAn the field test. Sujcjects Here randcily selected and 
assigned leSfcrship in cne of tuo equal size gicups having the saie 
nuaber of BalBb and females froa each grade. Data'were cttained by. 
clinical interview, PLOT, and intelligetce test records. Subjects iti 
group one received five clinical inter^ieis followed by fICT, while 
subjects in group t«o were administered the instruaents in reverse 
ordet. Exaaination of the findings led tc tic conclusions: (1) the 
construct validity of the group test was partially estatlished; (2) a 
learning effect was ptesent in the PLOT total scores which, was 
attributable to the^ previously adainistered clinical interviews, but 
no such effect was preisfnt, in general, in the clinical ieterview ; 
scores that were attributable to the adaini&tra'tion of PICT. 
(Author/HH) 
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' Abstract 

The problem of this investigation was tb answer the question: Can a group- 
administered test of Piaget's formal operational stage be developed and construct 
validated? A related problem involving a learning effect assbciated with Piaget's 
clinical methods ^s^^so inve|^ti gat^ The Piagetian Logical Operations Test , 
(PLOT), a group-ldministered i nstrumervtY was developed and field-tested t(f answer 
the question of this investigation. . 

Efghty-f our students in grades 10 - 12 of a south central* Indiana consolidated 
school corporation participated in the field test.. Subjects Were rancjomly selected 
and assigned fnembership in one of two equal size groups having the same number of 
^males and ferial es from each grade. Data, which was obtained by clinical interview, 
PLOT, and intelligence test records, was analyzed us|ng a 2.x 3 \ =2 factorial design. 
Subjects in groyp one received five clinical interviews fpllowe^ by PLOT while sub- 
jects in group two were administered the instruments in reverse \^der. A Campbell . 
and Fiske multitrait-multi method matrix consisting of three Wthods and four traits, 
factor analysis, *and three-way ANOVA were employed to statistically examine the date 
obtained. -^""^ 

^ <3 ... 

Analysis of data revealed several findings: (1) The internal consistency reli-' 
ability (alpha) of PLOT was ; 85. Reliability of individual sqales was also reported. 
(2) PLOT was significantly and substantially correlated with Piaget^s clinical 
method. (3) PLOT total scores and intelligence test scores did^not show high factor 
loadings on the same factor,. PLOT total scores and clinical interview total scores 
did not'exhibit high factor loadings on the same fa|ctor. (4) Subjects who were pre- . 
viously. administered clinical interviews scored significantly higher on PLOT than 
subjects who did not receive interviews prior to PLOT, but subjects wlio receive PLOT 

. ^ : ' ^ •.■ ■• ^ ■ • , / 

previous to th6 clinical interviews did not score significantly higher on the tojkal 
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clinical interview score than subjects who did not take PLOT prior to the clinical 
interviews. ' ' 
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Examination of the findings Ipd to two conclusions: (1) The construct validity 
of the group test was partially establish^^i2) A learning effect was present in 
the PLOT total scores which was at^tribu|^^^pthe previously administered clinical 
interviews, but no such effect was prej 



scores that was attributable to the^^previ 




fieral, in the clinical interview 



stration of PLOT. 



/ 



si 



; • Construct Validation. 

' ' ■•' . ■ ' . ■ 1 

Introduction ' 
Piaget employed the clinical inter^view 1>ecause that method pirovided the most,^--' 
useful framework for his research on the developiwent of cognitive thought within the , 
individual. However, educators who wish tp study cognitive development and Its 
implications for science tea'ching across individuals by the clinical jnethod encounter 

two m^pr drawbacks. One is the amount of time consumed, and the second is the 4hher- 

J" . - 

ent methodological nonstandardizations associated with the clinical method. 

Several workeips (Burney, 1974; Lawson, 1978; Longeot, ^963, 1964; Raven, 19^^; 
Renner, 1977; Shayer^'and Wharry, 1974; and TisHer, 1971) attempted the construction/ 
of a group administered measure of Piagetian cognitive development. One goal of these^ 
assess^nts was the modification of science teaching strategies for better consistency 
with the intellectual development of children. However, each effort only partially 
meets three criteria which seem prerequisite for a valid efficient test: (1) logical 
equivalence of written test items and the mental logic of specific Piagetian tasks; 
(2) evaluation of the reliability and construct validity of the group measure;^ (3) 
assessment in an efficient objective format of specific reasons offered by children 
in support of cognitive decisions. The goal of this study is to describe the develop- v 
ment and construct validation of atest which fulfills the aforementioned criteria. 

The ^Piagetian Logical Operations Test (PLOT) 

In tHHs sict ion several characteristics of the Piagetian Logical Operations Test' 

■ V . ■ • ' . ■ ■ ' 

(PLOT) are delineated, including format, items, scales, and scoring procedures. 
PLOT is. an objective multiple-choice test with'four alt^natives per question.and 
four individual scales: (1) conservation of volume by liquid displacement, (2) sep- 
aration and control of variables, (3) combinatorial analysis, and (4) proportional 
thought. The conservation scale represents a trait of late concrete thought proposed 
by Karplus and Lavatelli (1969). The three remaining scales each represent a trait 
of formal thought proposed by Piaget (Indelder and Piaget, 195^). £ach scale con- ^ 
sists of three item types, content c|uestions that assess the subject's comprehension 
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of a task, Ldecision "questions which require a' cognitive, deciaio^ by the student, and 
reason qu^tlons whjch identify reasons for cognitive decisions. At least one reason 
question is designed to specifically rate subject reasoning, patterns on each decision 
question. I All PLOT questions are similar to questions asked in clinical interviews, 
the principal difference being the format. Thus, the Vqgic necessary to answer the 
questions may be assumed identical to the logic required to so I've the corresponding 
clinical tasks. - ' ^ 

At least one cognitive task appraising each trait of Ptag^ian thought is pre- 
sented' via video-^ape. The same tasks were also' given by clinical interview, and . 
they are described in that section of the report.' Employment of video-tape demonstra- 
tions of Piagetian tasks permit the admini strati on 'of PLOT to classroom size groups 
(30. students ) and control of variation in administration procedures. Subjects observe 

\ \ ■ ■ ■ : ' ■■ ■ " " ■ 

the task and answer questions in ^ the appropriate section 'of %he test booklet. A PLOT 
total score and individual 'PLOT scale scores aj:|& available, and each score is calcu-' 
la ted by summing the number, of correct answersv-tn the appropri-at^" scale or the entire 
test. ^ y " ~ • " 

" . , Validation ProceduiTes • ■ ' 

the' procedures employed to evaluate the reliability and construct validity of 
PLOT are described in this segment. Included arie thraspects of construct validity, 
IrjstrumeRts, statistical procedures, and special problems associated with Piagetian 
measurement. 

Construct viklidation^s typically a twa dimensional process. .One .aspect, con- 
. vergence, is concerned with lustainment by independent measurement, and. the other 
dimension, descrlminance, is focused on the independence of tgsts not 'constructed to 
measure the sajne trai;ts (Nunnally, 1967). To examine both aspects of construct valid- 
ity, multiple trails and multiple methods must be empToyed (Campbell and Fiske, 
1959)./ ; , ' \ 
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The methods Aitilized irt this study'were PLOT, the Pi|igetian clinical 'interview, \ 
the Lorge-Thorndike' Intelligence' Test, (form 1,' Levels C-,D,E) and the^Cognitiv^e Abil- 
ities Test, (Form 1, Level Q)'.- .Traits measured were conservatioij of volumja by.'3iqul4 
displacement, separation and , control of variables, combinatorial analysis, propor- 
tional thougl^t, verbal, nonverbal , and quantitatiive abilities. - ^. / > * 
, The Lorge-ThtiK'ndike Intelligence Test and the Cognitive Abilities Test a,re 

measures of a general mental ability, the former having verbal and npavetbal scales., 

' J' ' " - " ••• 

and -the latter having verbal, nonverbal, and quantitative scalfes. A score for each ' 

* - ^ * •■• ^. 

scale was used as' wel 1 as a total score for each mental ability'f{«asure, the sm of 

scales scores for the G*A;T. and the mean of the scales for the L.T.LT. " 

F'ive Piagetian tasks were selected for administratiorl to subjects by clinical 

interview: (1) Volume of Metal Cylinder^ % Lic^uid Displacement (Karplus anfl Lava- • 

>' ' • ' ' ' ^ " ' 

tellL 1^9); (2) Flexibility of Bending Rods ("irthelder'and Pia^get, 1958); (3) Colored 

andl Colorless Chemicals (Inhelder and Piaget, 1958); (4) ,M'r. , Ta'lWMr. Short-Measurp-*^ 

ment with Paper Clips (Karf)lus and Lavatelli , 1969); (5) Equilibrium in the Balanfie 

(Inhelder and Piaget, 19^). Tasks 1,2, and 3 assess conservation of, volume, sepa- ^ 

ration* and control of variables, and combinatorial analysis, respectively wheijeas 

tests 4. and 5 measure direct aniJ inverse aspelts of proportional thoughts, resrac- - 

tively. . ' ^ ^ ^ \f \, I 

; . « Two evaluations of each clinical interview were made. First, a categorical 

(yes/no) decision concerning tf\e presence. of a mental schema was made.. Second, ^ 

- . ■. ^ ^ . ' • i ■ - ■ ■ ^ ■ ' 

series of behavior statements rjepreisent^'ng possible behaviors of subjects during 
Interviews were marked (yesxl/no^O) "-and, "total ed. Lists of behavlbr-oriented 'state-' 
melrits^ called^ befjavi or observation sheets , were previously' found to be \reliable and 
valid by Staver (^IsV) in measurement of ^Jageti an schema by. clinical interviews. . 
Evaluations of the inter-rater reliability and concurrent validity of behavior obser-^ . 
vation sheets employed irKtfWs research are discussed elsewhere (Stavef 1978). 
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. . - 7 . ... ^ - ' y V • if 

' ^ All cllaicaV interview fey^'uatidns were done by a* three- judge panel of 'advanced 
v^cifehcfi'edutatidi!! graduate students and ppst-doctoraK/esearch associates. 'Training 
; of eva)uator§ fricluded discussiSrl^f involved. schema, .clarification of behavior 

^- >v . ' • . . ■ • ' ' - , • 

5tat;ementSs an^l^ practice in , the use of^ the -behavior observation shee 
■ "° * . '' > _ , . ^ ■, 

, ' Employijient of seyera? measurement methods and assessment of several" traits of 

'. 'cognitive? thought,-, altljough'-n^i^essary for* evaluati(i)n of construct validity, can be- 

come •unwieldy^/'.A co^nvehierit way simplify the evaluation is to construct. a multi- 

. tra4t-muliimethQd mdtif^ix of the^dbrra^ The Campbell and Fiske (195a) model 

used in this stiidy prj^sejitatioii of all 'correlations among several traits. and 

methoc^ in matriXv^ortn^.for such, evaluation. 
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To Vurther. evaluate the construct validity of PLO'T', the scores of all instrume^nts 
' werev^^Me'ct to a' factor .analysii 5. f^estilts 0/ this procedure could provide additional 
evidehc\ for^privergenc^^ diVcriini nance. ^ The SPSS-Factor Program '(Nie, et al., 
1§75), ehploying the .pLri,ncipa^ c6^mponents method with iterations to achieve orthogonal 

'.factors and-^vai^imax rotation to simple structlire tff^all factors having eigenvalues' 

: • •■» « •■• u ' ♦ 1 • '" ■ t ^ ■ ■■ ■ 

grQatsSr-tKan hO, was used. ■ 

, " ' ." ' / 0 I ■ ' 

' ". Two' validation ..proble[]is remain to be delineated, the selection of a sample and /. 
■ th6 evaluatltfnt^df a learning: effect' associated jwith the clinical method. , A cj^itical 
»aspect',of tbe .validation procedure was the iselection of a subject sample from'a pop- 
ulatibrt' cohtaining substarijtial numbers^ of co transitional and formal thirikers. 

. Baseil upon =4Mappetta'*s (1976) review of stui^ies concerning the developmental levels 
r of, secbndary and coUege 'Studgijts, the conclusion was made that a random sample of 
'Senior high school subjects would provide the best mixture, Ther^efore twenty-on6 
males anrd.twenty-one females were randomly selected from each grade of a large 10-12 
jrade higfTschool irr a south tentral Indiana consolidated. school corporation. This • 

' •' ^ ' ' . i 

, selection procedure yielded a sample of 125 subjects which coatained equal numbers 

* I ■ ■ . : 

of 'males and- females within each grade. . . 
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The final validation problem waa a learning effect associated with the clinical 

method. Subjects often show more advanced reasoning patterns in the second inter- 

view when a clinical task is administered tW'jce within a brief time period. Such . ' 

learning effects could act tp decrease Qorrelations among Piagetian variables and 

give spurrioAisly fow estimates of convergent validity. To evaluate subject learning . 

effects, seven males and seven females within each grade Wjsre randomly assigned mem- 

bership in the cells of* a 2x3x2 factorial design (Kirk, 1968) and a replacement ^ 

group involving two groups, three grades, and two sexes. No pretest^was employed 

because of the reactivity of Piagetian measures. Further, Campbell and Stanley (19B3) 

if ■ - ' 

maintain t^t the most adequate assurance concerning the absence of initial bias 

between groups is randomization. Treatment was considered to be the administration 

of the five clinical tasks'in order. li 2, 3, 4, 5, and PLOJ was considered to be the 

posttest. The 42 students comprising group 1 were given treatment before posttest 

evaluation whereas an equal number of subjects in group 2 were administered the post- 

test followed by treatment. Thus, each group acts as a control for its counterpart. 

The 42 subjects In group 3 formed a replacement pool. Children failing to parti'ci- 

pate in the first activity of their assigned group were replaced by a randomly chosen 

subject whose grade and sex matched that of the lost subject. No student who failed ^ 

to continue after participating in the initial activity was replaced. 

Findings, Conclusions, arid Discussion 

Validity of PLOT ' , ^ ^ ^ 

To evaluate the reliability and construct va*lidjty of PLOT, information derived 
from the corraTational matrix; factor analysi^s, the learning effect, and the effi- 
ciency of PLOT is set forth in succeeding parts of this section. 

The correlations among the-three methods and four traits are assembled into a 
Campbell and Fiske mat^iix dnd presented fn Table 1. A detailed* inspection of the . 

- (Insert Table 1 about here) ^ 
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matrix is necessary to determine the findings. The internal consistency reliability 
(alpha) value for each instrument scale is shown as the value in parenthesis. For 
. example alpha=.85 for the PLOT conservation scale. Alpha for PLOT total score, not 
shown in Table i; is also .85. According \o criteria set forth by Davis (1964) for 
individual differences measurement, the reliabilities of PLOT scales 1 and 4, and 
the tot^al score are acceptable whereas alpha for-PLOT scales 2 and 3 are insufficient. 
Four criteria are examined in Table 1 to determine the validity of PLOT, first, 

o 

correiatiins the same trait measured by different methods should, be significant 
and substantial- These correlations form three diagonals xa lied validity diagonals 
and the entries are all underscored. Seven of the twelve validity diagonal values 
are significant and substantial,^ thereby indicating convergence among the methods for 
those tracts. Second, measures of the same trait should eochibit higher ppsitive ^r- 
reOations between each other than with measures of different traits employing 
different methods. This means that a validity diagons^l entry in Table 1 should, be 
greater than values in its row and column of the adjacent heterotrait-heteromethod 
triangles (enclosed by broken lines). Inspection of Table 1 for the seven signifi- 
cant validitjc^^di agonal cases shows the second criterion fulfilled in only two cases, 
•B^B2 and DjD2. Third, measures of the same trait should show higl^er positive cor- 
relations between each other than with measures of different traits using the same 
method. With respect to Table L, the validi-ty diagonal value for a variable should ' 
be higher than its values in the heterotrait-monomethod triangles (enclosed by solid 
lines). Examination shows that only one significant validity case, 0^02, meets this 
criteria. "^Fourth, measures of different traits should exhibit an identicail pattern 
of intercorrelations, among each other across heterotrait-rnonomethod and heterotfait- 
heteromethod triangles. Such a pattern in Table 1 would be a single trend in the 
magnitudes of correlations for'all .triangles. In fact, no single pattern or trend 
is detected. The last three criteria are focused on the discriminant Sspect <3*^- 

ERIC • IC 
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struct validity, and analysis 'of Table 1 shows little evidence of discriminant vali^T- 
Ity for* the Piagetian and general intelligence measures. 

Factor analysis represents an additional method for the evaluation of the con- 
struct validity of PLOT^ and the results of PLOT; clinical interviews, and the 
mental ability tests are shown in Table 2. Convergent validity between variables is 
exhibited by high loadings for variables on the same factor whereas discriminant valid 
ity among variables is supported by high loadings coupled with modest loadings on the' 
•same factor (modest-high couple). In thi's study a high loading is -.60, ^a medium load 
ing is -.40 and -.59, and a low loading is'-. 39. inspection of Table 2 reveals that 
high loading on the same factor are not observed for PLOT total score and all ,PtOT 
scale scores with the corresponding total clinical interview score and clinical task 
scores. Therefore, little evidence for convergence between the two Piagetian methods 
is present. Modest-high factor loading couples for PLOT total scores, and PLOT 
scales, 1, 3,, and 4 with intelligence test scores are observed whereas ott^^ half the 
modest-high couple is seen for PLOT scale 2 with' intellegence measures.; PtOT scale " 
2 exhibits a medium factor loading on factor 1- which exhibits high loadings for intel- 
ligence measures. Thus, substantial evidence for discriminant validity between 
Piagetian and general intelligence measures is found, but little evidence for con- 
vergence of the two Piagetian measures is observed. 

The correlational and factor analytical findings present an enigmatic situation 
which requires discussion. The correlational analysis provides evidence only for* 
convergence. between Piagetian measures whereas the factor analysis provides rather 
clear evidence for discriminance, but little support of con ver^nce is found- The 
correlation between the PLOT total'score and the total clinical interview score, .59, 
is compara^ble with higher validity diagonaV values' in Table 1 a'nd- further supports 
convergence. However, the lowest correlations in the heterotra^it-heteromethod trir . 
angles suggest that the measurement methods in tbis research are not entirely inde- 
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pendent/ PLOT. and the cjinical iri^rvlew metiwd share cdninbn ri^terUls, tasks,. and. ^ 
questions. Principal differences ^ re dempnst^^ versus mafiipulatijpn of. materials, 
•farced multiple-choic? versus bpen-epded que^tTon-answer format, afid- written versus 
oral response. PLOT arid the mental ability tests\ share a. common question-answer, for- 



mat and the necessity of reading for comprehension, Theriefore,' it is probable that 
all three methods are related. Additionally, the traits themselves may form a uni- 
fied system of thought and are not com(J.letely independent, Campbell and Fiske (1959) 
maintain that some evaluation off val^idity can^ be made' in this sitliation, and accord- 
ingly, some convergence 'i 5* indicated for PLOT;scales -2 and 4, and the PLOT total 

. ■ ■ ^1 ■ 

scores. , v • ' * ' . 

A \ \ ' ' •■,*.. 

With respect to; the factor arifflytical procedures, little or no evidence of con- 

vergent -validity is found by .observation/of high-highnoading couples on the same' 

■ ^ * • ^ ' \ / ■ f ■ ■ ■ * ' ' • ■ 

factor for PLOT and clinical interview variables. Furtherv substantial results indi- 

cat-ing discrimitfant valrdity are present in the lotiding ^patterns of PLOT and mental 
ability variabnes. The loadi nig patterns permit both the identification of rotated- 
factors ^d the es"^lishment of discriminant validity for PLOT by this method. 

In the factor .solution presented in Table 2 only mental abil ity variables exhibit 
high loadings on factor one', ^^emaining variables load modestly with one notable^ 
exception, PLOT part 2, on this factor; it shows a -loading -on factor one of .53, 
medium. . Factor one is clearly identifiable as a factor associated with geneigal intel- 
ligence. PLOT part 2 loads substantially on this factor because the abil^ity to . 
sepaf^ate and control variables seems to be associated \9ith general mental ability. 
This factor also account^ for 76-4% of the total variance.- ' 
. . Factor twa, wh-^ch accounts for 13.3% of the^ total variance, is somewhat more ^ 
^ difficult to identify. Inspection of factor two reveals high loading for fiVe of the 
.ten clinical interview yariabTes whereas one of the remaining, five, variabiles shows'a j 
medium loading and the other four exhibit modest loadings' on factor two. AlLinteT- • 
ligence varjables loact modestly on this factor as well as do all PLOT variables.^ 
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Analysis of. factor three, accounting for 10.3% of the ^tal variance, furtHer aids in 

the identification of factors two and three. All 'pLOT- variables exhibit high' loadings 

on factor three except scale 2 which loads • 53, .medium. Further, all ngotal ability 

and clinical interview variables are observed to load modestly on factor three. 

What seems to have occurred in the rotation to simple structure is a variable separ- 

ation on orthogonal factors by method/ Factor one, , as previously identified, is ^ 

associated with general "meptal'^ability, Factor two*, although less clearly '?o,. seems 

related to Pi ageti an cognitive development assessed through xlinical interviews 

whereas factQr three is. revealed to be connected, with PLOT'aS a measurement method 6^' 

Piagetiair-cognitive d'evelppment; Although the factor solution gives ample evidence 

of discriminant validity, i.t also 'yields little suggestion of cdnvergente for PLOT 

and the clinical method: Therefore, it is concluded that convergent and discriminant 

validity are partially established. 

Learning, Sex, and Grade Effects 

Three-way analyses of variance were performed on the' PLOT and clinical interview 
scores, and the findings,, conclusions, and discussions of the l^rning phenomenon, 
plus grade and sex effects are set forth in this section. 

^ignificant differences in favor of the group which was previously administered 
the series of clinical interviews exists in the PLOT total, scale 1, and scale 3 
mean scores compared to the group which did not receive clinical interviews prior to 
PLOT administration (F=i2.06,. 15.90, 6.53, respectively; p<.05, df=l,55). Group mean 
differences for PLOT scales 2 and 4, although in favor of the group receiving prior 
cTini'cal interviews, were not significant. Gradually increasing mean scores for PLOT 
and its individual scales were detected with increasing grade levels but mean differ- 
ences were not significant. Also, no significant differences with respect to sex 
were revealed for PLOT and its scales and no significant two and three-way inter- 
actions were presient. ' v ^ 
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Group mean differences on the clijiical interview variables, although favoring 
the^'grbup refceiving PLOT prior to the interviewT^ere generally not significant. 
exceptions were the significant differences on clinical tasks 1 and 2 {F3l2;84, 6.96, 
respectively; p<.05, df»l,60; 1,58', respectively). An increase in grade level is . 
generally accompanied by an increase in the mean for. the total clinical score and ^ 
all task scores, but only the mean difference for" task 3 is significant {F=3.94; 
p<.0*S, df=l,56). The mean differences' for sexes were not significant, and no sig- 
nificant two or three-way interactions were detected. . 
* _Consi deration of the findings concerning Teaming effects leads to several con- 
; elusions. First, a learning effect attributable to the prior administration of clin- 
/ ical int:erviews is present in the PLOT total score and PLOT scales 1 and 3. Second, 
the; clili^cal interviews, when considered as treatment, have a similar result across jth'e- 
main efJ'ects of group, grade, and sex taken in pairs, or in triplet. Third, a Team- 
•ing effect attributable to prior PLOT administration is present only In the scores 
of clin1ca|^ tasks 1 and 2. It is not present 1n any remaining clinical variable 
Including the total score. Fourth, PLOT, when viewed as treatment, has a similar " 
effect across the main effects of group, grade, and sex taken in pairs or in triplet. 

^ The general presence of , a learning effect in^LOT scores attributable to the 
pVior administration of clinical interviews, and the general absence of such an 
effect in the clinical scores due to prior PLQJ administration presents another enig- 
'matic situation. The two instruments are designed to measure -the same traits, and 
they have coninon materials and similar questions. A plausible explanation -arises ^ 
from the theory itself, and Piaget's thoughts on the self-regulation mechanism. 
Prior to the onset of formal operations, and still valuable In formal thought, is 
the active manipulation of the environment by thechild. A fundamental difference 
between PLOT and the clinical tasks is that during Interviews subjects activeTy manip- 
ulate the materials whereas such objects are only observed on video-tape on PLOT. 
This manipulation-observation difference seems important In accounting for the' 
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general presence of learning . associated with the clinical method .and its general 
absence in PLOT. The isolated cases of learning in clinit^aj tasks 1 aind.2 attribut- 
able to prior PLOT administration are most probably explained by.shared trait and - 
method variance. . / [ : , 

Ex Post Facto An alysis 

. Data analysis, including: item analysis of PLOT, indicated three areas for, ex 
post facto examination of the data conc^erning' the reliability and construct validity 
of PLOT. The areajs are correction foi/ attenuation in correlations, deletion of PLOT 
content questions, and evaluation of PLOT decision a*nd reason scales. 

The correctio/»' for attenuation procedure (Guilford and Fruchter, 1978)' was 
applied to each entry in table 1/ to determine the extent of the detrimental effect . 
exerted. on the validity of PLOT;/ by the low reliability coefficients for PLOT scales 
2 and 3. Although the unattendjtted correlations wer? higher, especially entrie/ in 



the validity diagonals, no neW information about the construct validity of PLOT was , 
yielded by the analysis method described earlier. Therefore, it was concluded that 

> " ■ ■ ■ V' ■• ' ■ ■ " ' ■ ■ 

while the low reliabilitii^s/of the two PLOT scales are, detrimental, they are oot the 
primary problem'tn establi^hingHhe construct validity of PLOT. . 

Three kinds jf items, content, decision, and reason, compose Pioi. Item analy- 
sis revealed that Students obtained a mean of 11.97 and a standard dfeviatidn of i.O? 
on the thirteen content questions whereas their performance was much more diverse on 



the decision and 



reason questions. The deletion of content questions from the PLOT 



total score represents an approximate linear transformation/^Such transformations 
have no effect on correlation among variables; (Hopjcins and Glass, 1978) .thus, removal 
of content questions has little^ effect on construct validity. Deletion of PLOT con- 
tent questions d-^d' yield a slight positive trend in reliability. Such, questtons 
could be deleted from the entire test, but some discussion is justified concerning 
the role of cpntent items in the measurement process. 



One lineiof thought 
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is to remove such items from the test altogether because they 
serve no other function than to increase the score. A second direction for |onsi der- 
ation is to reiijove the content items or^ly from the ^co re because the function of 
such items Js lip focus the subjepts' attention on the most important aspects of the 
problems to be l|olved. This point is c^ruclal because the subjects only view a demon- 
stration of the problems; materials' are not handled. Therefore, the presence of 
such items may be critical to the subjects' comprehension, or th^ pwbl^a^ and 
thereby influlnces ?inswers to decis,ion and reason questtons\ /Jhe fact that most sub- 
jects receive near perfect scope/merely indicates that the goal for which the i 
.questions are designed is being achieved. Thus, a student's score on the content 
items provides little indication , of cjjrrent developmental level., That information 
is yielded through answers to decision and reason qliestioRS in each scale. / 
A tBird direction of exrposft facto analysis seemed justified. The concepts 
'decision' and' * reason 'were <;oris*idered as traits measured by /the three aforemen- 
tioned methods in a new Campbell and Fiske* matrix and the correlations appeal^ in 
fable 3. ' . : • 



(Insert "(able 3 about here) 
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^Analysis of Table 3 by methods. outlined earlier revealed substantial evidence 
for convergence among all the methods (all validity diagonal entries are significant 
and most are substantial), but little information concerning discriminance. Thus, no 
new findings were uncovered, and the previous discussion of the matrices holds. In 

summary, ex post facto.analysis yielded no information which conflicted with earlier 

'. . ' ■ \ ■ • \' ■ ■ " 

results. • 

Efficiency and Practicality of PLOT \ i 

One requirement cited earlier for a useful Piagetian test was' the development .of 

ari efficient practical measure. . An iij^ortaM characteristic of PLOT as an unttmed ^ 

•• • •• / ■ ■ • ■ ' "I ■ ■ • ■ • 

test iis thal^ eachvgroup proceeds through the sequence of video tape demonstrations 

and written (i|^uestions at tHe^ p^ce o/f thiS slowest student. Data concerning time 

IE ■ ■ • . ' • 
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required for administration of PLOT^to subject show that PLOT was /dminiistered 23 ' 
times with a mean administration time of 46, 1 itiinutes and' a range of 36 - 56*mihutes. 
These data are indicative of the fact that PLOT can ^^e adm1nist|fred to Ijidivi duals 
or sma.ll groups of students within a 55-minute period. ^- I ^ T ^ 
/ ^ Implications for Teachers / 

The development , and construct validation of P|-OT, a group measure for assessing 
four Piagetian schema associated with formal thought was reported in this paper. PLOT 
was developed for use by. science teachers and researchers ^ in science education inter- 
ested in the assessment of developmental reasoning^ capabilities of students. One 
goal of science teaching is to match instruction- and. curriculum -materials with the 
^developmental level of the learner. Learning difficulties of students in middle and 
secondary school science have often been. attributed. to an inability to grasp concepts 
in science^ A more refined line of thought suggests that some students are not yet 
using reasoning patteVns requi red <.to comprehend certain' science concepts. Further- 
more, many concepts in science may be. taught in a manner consistent with either for- 
mal or concrete thought. However, a prerequisite to the matching process is a 
reliable, valid, efficient and practical measurement device. Although further test 
development of PLOT is appropriate, the prepondierance of evidence suggests that PLOT 
is a reliable, valid, efficient, and practical measurement tool, and may thus be 
employed by teachers and researchers for the aforementioneji purposes. 
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I , ."lO ..'U\ J8 \.02 I . ,..07 .2rs(.79) 



Aj K ,11 \62 M .22 I ^.20 \.« .22 
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Cj I .19 .58\ .56^\.4l1 I .28 .« s^'^ ^d 
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Note: Validity diagonals are the three sets of mderlined values. Reliability diagonals are the Uuwf sets of values in parentheses. 

Eadi heterotrait-ROK^Btiwd triangle ^1^ Each heterotrait-heteraiBthod trlmile is enclosed by a 

. broken line. ' . " • ■ 



*8l?iific»it, p < .05, cna-tailed test. 
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TABLE 2 • 
ROTATED FIVE FACTOR PAHERN OF PLOT, CLINICAL IlfrJliVIEW. 
AND MENTAL ABILITY TEST SCORES 



&»■■■' » ' ■ ■ ■ 
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" ' Factors 


Variable 


1 


■ 2 


3 4 ; 5 Commuriality 



Non Verbal IQ 

Verbal iQ 

TotallQ 

CAT-Vgrbal 

CAT-Quantitative 

CAT-Non verbal 

CAT-Total 

Categorical Decision 
Categoricel Decision 
Categorical Decision 
Categorical Decision 



1- . 

2 

3 

4-5 



Total Categorical Decision 
Task 1 , . ' 

Task 2 ■ 
Task 3 
Task 4-5 

Total CI initial Interview 
PLOT-Part If 
Part 
Part 3 
-Part 4 
PLOT-Tota^ 





.84 


.14 


.00 


. 17 


.10 


77 


68 


.78 


.06 


.25 


.18 


.21 , ■ 


.74 


68 > 


.92 


. 10 


.14 


.19 ' 


.17 


' .93 


68 


.87 


.13 


.09 


.17 


.15 


.84 


-70 


.71 


.22 


.15 


-.03 


;43 


.75 


70 


.80 


.23 


.29 - 


-.04 


.08 


.79 \ 


70 » 


.91 


.22 


.19' 


.03 . 


' .25 


.97 


70 


.05 




.23v 


.05 


.07 . 


.76 


70 


.39 


.58 


.17 


.31 


.15 


.63 . 


68 - 


.12 


.11 


.05 


.85 


.13 


.78 


66 ' 


.31 


•.14 


.08 


.07 


.84 


.83 


70 


.37 


.60 


.18 


.49 


• .46 


. 9^9 


66 . 


.10 


.84 


.07 ' 


-.05 


.10 


■ .73 


66 


.35 


- .67 


.11 


.33 


.13 


.71 


72 


.09 


.09 . 


,.10 


.81 ; 


-.11 


'.69 


70 


.35 


>22 


.23 


.01 


,82 


.91 


68 


.38 


.62 , 


.20 • 


.51 


.39 


.99 


66 


.03 


.01 


.70 


.14 


.00 


.51 


66. 


.53 


.15 


.53 


.38 


.13 


.74 


66 


.39 






-.14 


.05 


.58 


66 


.07 




\.66 




v38 


x69 


66 


.31 


.22 


.92 


.18 
— ^ 


1.00 


66 


.67 ^ 


2.31 


1.79 


1.52" 


1.08^ 







Eigen value 



lue 
)/v 
accounted for 



Percent oF variance 



6L4 



13.3 10.3 



8.7 



6; 2 



NOTEt Principal components analysis with iterations, varimax factor rotation, 
and pairwise deletion of missing data was employed. The number of 
rotated factors was limited to five. 



! TABLE 3. : 

,«■■'■ 

■ •» • . . . • ' 
. mmm m tie decision and reason characteristics of FORHAL THOUGlt HEASUREO By THE PIAGETIAH 
LOGICAL OPERATIONS TEST, CLINICAL INTERVIEHS, AHD TIE lilRGE-THORNDIKE.INTELLlGEHCE TEST 





. * 
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7\ : PLOT' 

r*— ^-t ^ 




Clinical 
Interview 


Lorge-Thomdike 
Intelligence Test 






h 




:A3 '63' . 


PLOT V ' ' ' ' 

r 


Decision Ai (.61) 
'Reason Bj .78 ' I 


m 


. , 


K ■ . ■' 


■ ♦ 

1 Clinical 
y -.Interview ' 


Decision Aj, ' .56* 
Reason B2 > IT 

» 


.39 


.63 (.74) 

■ \ 


r 


Lorge-Thomdike 
; Intelligence Test 

■ ■ ■■.>■'■.' 


Verbal' A3 ' 
Nonverbal B- .37 

■ • 


.45 ' 
J! 

( 

1 


!<)■ ^ 

, ;57* .48 
,.51 ..45^, 


/(.90) 
.80 (.91) 
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*Note: Significant, p < .05, one-tailed test , 
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